Skip to content

recipe(cross-encoder/ms-marco-MiniLM-L4-v2): add text-classification fp32/fp16 recipes#1121

Open
codykk wants to merge 1 commit into
mainfrom
yongyue/add-cross-encoder-ms-marco-MiniLM-L4-v2-recipe
Open

recipe(cross-encoder/ms-marco-MiniLM-L4-v2): add text-classification fp32/fp16 recipes#1121
codykk wants to merge 1 commit into
mainfrom
yongyue/add-cross-encoder-ms-marco-MiniLM-L4-v2-recipe

Conversation

@codykk

@codykk codykk commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Recipe-only contribution (Effort L0). Adds fp32 and fp16 recipe configs for
cross-encoder/ms-marco-MiniLM-L4-v2 (BertForSequenceClassification, 4 layers, task text-classification)
on two verified EP/device combinations: CPU and DML (GPU). Goal L1 (perf) PASS on all 4 configs.


1. Recipe path(s)

  • examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/cpu/cpu/text-classification_fp32_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/cpu/cpu/text-classification_fp16_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/dml/gpu/text-classification_fp32_config.json
  • examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/dml/gpu/text-classification_fp16_config.json

2. README row

N/A — README not modified in this PR.

3. Build output dir

  • temp/minilm4_cpu_fp32/
  • temp/minilm4_cpu_fp16/
  • temp/minilm4_dml_fp32/
  • temp/minilm4_dml_fp16/

4. Build log

  • cpu/cpu fp32: ✅ Build complete in 16.2s (Export 3.8s, Optimize 12.0s)
  • cpu/cpu fp16: ✅ Build complete in 16.9s (Export 3.5s, Optimize 12.2s, FP16 0.7s)
  • dml/gpu fp32: ✅ Build complete in 16.5s (Export 3.7s, Optimize 12.3s)
  • dml/gpu fp16: ✅ Build complete in 17.3s (Export 3.8s, Optimize 12.5s, FP16 0.7s)

5. Appended findings

N/A — recipe-only L0 contribution.

6. Optimum-coverage probe

bert architecture fully supported by Optimum and winml. winml inspect confirms all components at "Default" status.

7. Claimed (Effort, Goal, Outcome)

Axis Tier
Effort L0 (recipe-only)
Goal L1 (build + perf)
Outcome L0 (recipe + report)

8. Goal-ladder verdict table

Tier Verdict Evidence
L0 (build) PASS All 4 configs build successfully
L1 (perf) PASS All 4 configs produce valid latency/throughput (see item 10)
L2 N/A Goal ceiling = L1
L3 N/A Goal ceiling = L1

9. Methodology-evolution declaration

No methodology friction observed.

10. Perf & eval data

EP / Device Precision Verdict Throughput RAM Δ VRAM Δ (local)
OpenVINOExecutionProvider / cpu fp32 PASS 82.69 samples/s +159.30 MB
OpenVINOExecutionProvider / cpu fp16 PASS 85.10 samples/s +160.10 MB
DmlExecutionProvider / gpu fp32 PASS 81.25 samples/s +283.00 MB +171.30 MB
DmlExecutionProvider / gpu fp16 PASS 31.42 samples/s +243.20 MB +101.00 MB
QNNExecutionProvider / npu * HOST-BLOCKED No NPU hardware

11. Component / op-level data

  • Architecture: 4 hidden layers, 12 attention heads, hidden size 384
  • Fusion patterns applied: gelu_fusion, matmul_add_fusion

12. Reproducible commands

# Build (cpu/cpu fp32)
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/cpu/cpu/text-classification_fp32_config.json -m cross-encoder/ms-marco-MiniLM-L4-v2 -o temp/minilm4_cpu_fp32

# Build (cpu/cpu fp16)
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/cpu/cpu/text-classification_fp16_config.json -m cross-encoder/ms-marco-MiniLM-L4-v2 -o temp/minilm4_cpu_fp16 --precision fp16

# Build (dml/gpu fp32)
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/dml/gpu/text-classification_fp32_config.json -m cross-encoder/ms-marco-MiniLM-L4-v2 -o temp/minilm4_dml_fp32 --ep dml --device gpu

# Build (dml/gpu fp16)
winml build -c examples/recipes/cross-encoder_ms-marco-MiniLM-L4-v2/dml/gpu/text-classification_fp16_config.json -m cross-encoder/ms-marco-MiniLM-L4-v2 -o temp/minilm4_dml_fp16 --ep dml --device gpu --precision fp16

# Perf
winml perf -m temp/minilm4_cpu_fp32/model.onnx --device cpu --iterations 20
winml perf -m temp/minilm4_cpu_fp16/model.onnx --device cpu --iterations 20
winml perf -m temp/minilm4_dml_fp32/model.onnx --device gpu --ep dml --iterations 20
winml perf -m temp/minilm4_dml_fp16/model.onnx --device gpu --ep dml --iterations 20

@codykk
codykk marked this pull request as ready for review July 16, 2026 04:05
@codykk
codykk requested a review from a team as a code owner July 16, 2026 04:05
@codykk
codykk force-pushed the yongyue/add-cross-encoder-ms-marco-MiniLM-L4-v2-recipe branch from 3f3bf46 to 3bb2f8b Compare July 16, 2026 04:15
…MiniLM-L4-v2

Recipe-only contribution. Adds fp32 and fp16 recipe configs for
cross-encoder/ms-marco-MiniLM-L4-v2 (BertForSequenceClassification, 4 layers)
on two verified EP/device combinations: cpu/cpu and dml/gpu.

Goal ladder (verified on local hardware):
- L0 (build): PASS on all 4 configs
- L1 (perf):
  - cpu/cpu fp32: 82.69 samples/s
  - cpu/cpu fp16: 85.10 samples/s
  - dml/gpu fp32: 81.25 samples/s
  - dml/gpu fp16: 31.42 samples/s
@codykk
codykk force-pushed the yongyue/add-cross-encoder-ms-marco-MiniLM-L4-v2-recipe branch from 3bb2f8b to 9e36bf4 Compare July 17, 2026 01:29
@ssss141414 ssss141414 added the model-scale-by-skill Model support PR created or maintained by the adding-model-support skill label Jul 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

model-scale-by-skill Model support PR created or maintained by the adding-model-support skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants